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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
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Status 

1 )M Responsive to connmunication(s) filed on 23 February 2004 . 
2a)[2 This action is FINAL. 2b)n This action is non-final. 
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closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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6M Claim(s) 1,3.4,10.31.32 and 35-37 is/are rejected. 
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application from the International Bureau (PCT Rule 17.2(a)). 
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14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
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DETAILED ACTION 



1 . This action is responsive to communications: Amendment, filed on 2/23/2004. 
This action is final. 

2. Claims 1, 3-4, 10, 31-39 are pending in this application. Claims 1, 10, 31, 33 and 
38 are independent claims. In the Amendment, filed on 2/23/2004, claims 1, 10, 31, 33 
and 38 were amended. 

This application claims foreign priority dated 3/26/1999. 

3. The present title of the invention is "Image processing method and apparatus, 
and storage medium" as filed originally. 

Claim Rejections - 35 USC § 103 

4. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

5. Claims 1 , 3-4, 1 0, 31 -32 and 35-37 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Chen et al. (5,613,048) in view of admitted prior art and Hrytzak et al. 
(5,327,257). 

6. As per claim 1 , Chen et al., hereinafter Chen, an image processing apparatus for 
generating an image of a virtual space according to a user's operating using ray space 
data comprising: 

a recording unit adapted to record the ray space data ("selected views of a 
scene, from different viewpoints, are recorded", column 3, line 53-54); 
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an inputting unit adapted to input a user's operation of movement in the virtual 
space (Figure 2 18 movement control; "The movement of the virtual camera, to present 
different views of the scene, is controlled by the viewer through a suitable input device 
18", column 3, line 40-42); 

a setting unit adapted to set a sampling rate for the ray space data read out from 
said recording unit according to the user's operation, the sampling rate indicating a 
distance between pixels to be sampled ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicates the sampling rate is a function of 
distance between pixels to be sampled); 

a reconstructed unit adapted to read out ray space data from said recording unit 
according to the sampling rate set by said setting unit, and to reconstruct an image of 
the virtual space, the read out ray space data being arranged on a line in the ray space, 
and being on the line at an interval in accordance with the sampling rate ("Once each of 
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these views has been stored, the image from any viewpoint between any two of the 
recorded locations can be obtained by interpolation of two adjacent stored images", 
column 3, line 60-64; "In the example of Fig. 1, the movement of the virtual camera is 
one-dimensional", column 7, line 29-30; one-dimension inherently means a line; since 
the data is stored in accordance with a sampling rate, it is inherent that the read-out 
data is related to the sampling rate). 

Chen discloses an apparatus of generating an image of a virtual space. It is 
noted that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
disclosed a ray space theory has been used to describe a virtual space (page 1 , line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation unit adapted to interpolated pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-11. 
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Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose an apparatus of generating an image of a 
virtual space using ray space theory and Hrytzak discloses the generated image can be 
interpolated into a desired size in order to maximized the view of the interpolated image. 

7. As per claim 3, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the moving 
speed in a virtual space designated by the user ("The movement of the virtual camera, 
to present different views of the scene, is controlled by the viewer through a suitable 
input device 18", column 3, line 40-42, since the morphing, thus the sampling rate, is 
determined by the correspondence between two images and correspondence depends 
on the moving speed of the camera). 

8. As per claim 4, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 
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9. As per claim 10, Chen discloses an image processing method for reconstructing 
an image of a virtual space according to a user's operating using ray space data 
comprising: 

an inputting step of inputting a user's operation of movement in the virtual space 
(Figure 2 18 movement control; "The movement of the virtual camera, to present 
different views of the scene, is controlled by the viewer through a suitable input device 
18", column 3, line 40-42); 

a setting step of setting a sampling rate for the ray space data read out from said 
recording means according to the user's operation, the sampling rate indicating a 
distance between pixels to be sample ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicate the sampling rate is a function of 
distance between pixels to be sampled); 
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a reconstruction step of reading out ray space data from the recording means in 
accordance with the sampling rate set in said setting step, and reconstructing an image 
of the virtual space, the read out ray space being arranged on a line in the ray space, 
and being on the line at an interval in accordance with the sampling rate ("Once each of 
these views has been stored, the image from any viewpoint between any two of the 
recorded locations can be obtained by interpolation of two adjacent stored images", 
column 3, line 60-64; "In the example of Fig. 1 , the movement of the virtual camera is 
one-dimensional", column 7, line 29-30; one-dimension inherently means a line; since 
the data is stored in accordance with a sampling rate, it is inherent that the read-out 
data is related to the sampling rate). 

Chen discloses a method of generating an image of a virtual space. It is noted 
that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
disclosed a ray space theory has been used to describe a virtual space (page 1, line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation step of interpolating pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
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discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-1 1 . 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose a method of generating an image of a 
virtual space using ray space theory and Hrytzak discloses the generated image can be 
interpolated into a desired size in order to maximized the view of the interpolated image. 
1 0. As per claim 31 , Chen discloses a computer-readable storage medium for storing 
a program which makes a computer function as an image processing apparatus for 
generating an image virtual space according to a user's operating using ray space data 
recorded in a memory, the program comprising: 

an inputting step of inputting a use's operation of movement in the virtual space 
(Figure 2 18 movement control; "The movement of the virtual camera, to present 
different views of the scene, is controlled by the viewer through a suitable input device 
18", column 3, line 40-42); 

a setting step of setting a sampling rate for ray space data read out from said 
recording means according to the user's operation, the sampling rate indicating a 
distance between pixels to be sample ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
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correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicate the sampling rate is a function of 
distance between pixels to be sampled); 

a reconstruction step of reading out ray space data from the memory according 
to the sampling rate set in said setting step, and reconstructing an image of the virtual 
space, the read out ray space data being arranged on a line in he ray space, and being 
on the line at an interval in accordance with the sampling rate ("Once each of these 
views has been stored, the image from any viewpoint between any two of the recorded 
locations can be obtained by interpolation of two adjacent stored images", column 3, 
line 60-64; "In the example of Fig. 1 , the movement of the virtual camera is one- 
dimensional", column 7, line 29-30; one-dimension inherently means a line; since the 
data is stored in accordance with a sampling rate, it is inherent that the read-out data is 
related to the sampling rate). 

Chen discloses a method of generating an image of a virtual space. It is noted 
that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
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disclosed a ray space theory has been used to describe a virtual space (page 1, line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an Image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation step of interpolating pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-1 1 . 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose a method of generating an image of a 
virtual space using ray space theory and Hrytzak discloses the generated image can be 
interpolated into a desired size in order to maximized the view of the interpolated image. 
11. As per claim 32, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and admitted prior 
art further discloses wherein ray space data is managed in a (x,u) space, wherein "x" 
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represents a position where he light ray intersects the X-axis and "u" represents a 
tangent of an angle the light ray nnake with the Z-axis (page 2, line 19-23). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

12. As per claim 35, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 

13. As per claim 36, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 
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14. As per claim 37, Chen, admitted prior art and Hrytzal< demonstrated all tlie 
elements as applied to the rejection of independent claim 10, supra, and admitted prior 
art further discloses wherein ray space data is managed in a (x,u) space, wherein "x" 
represents a position where he light ray intersects the X-axis and "u" represents a 
tangent of an angle the light ray make with the Z-axis (page 2, line 1 9-23). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

Allowable Subject Matter 

15. Claims 34 and 39 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

16. Claims 33 and 38 are allowed. 



Response to Arguments 

17. Applicant's arguments filed on 2/23/2004 have been fully considered but they are 
not persuasive. 

As per claims 1, 10 and 31, applicant alleges that in Chen, image resolution is 
always constant. In reply, examiner does not see applicant provides the basis of this 
argument. Applicant further alleges Chen's sampling rate is performed on a time axis. In 
reply. Examiner considers Chen also teaches spatial sampling (In the example of Fig. 1 , 
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the movement of the virtual camera is one-dimensional ... can be multi-dimensional", 
column 7, line 29-33). 

Conclusion 

1 8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Inquiries 

1 9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan Yang whose telephone number is (703) 308- 
6133. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi, can be reached at (703) 305-4713. 
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Any response to this action should be mailed to: 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 



or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (703) 305-47000377. 



Ryan Yang 
April 28, 2004 




